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ABSTRACT 

If learners are to engage in science inquiry, they need 
significant support, or scaffolding, to help them mindfully do the cognitive 
science tasks that are just out of their reach. One approach for supporting 
learners is to design computational tools that incorporate scaffolding 
features to make new practices accessible and visible so learners can engage 
in and understand these practices. In order to explore the systematic design 
and assessment of scaffolding features for software tools, the authors 
developed "Symphony, " a comprehensive scaffolded work environment 
incorporating process scaffolding features for ninth grade science students 
performing environmental science investigations. The research questions asked 
what conceptual process scaffolding strategies would support learners and how 
can those scaffolding strategies be realized in software. The scaffolding 
assessment involved three broad steps: identifying specific episodes where 
students used scaffolds to perform different inquiry activities; assessing how 
students used different scaffolds in each episode using a new set of 
assessment criteria; and summarizing the assessment information for each 
scaffold to describe how students used the different scaffolds over time. By 
analyzing how learners worked with the Symphony scaffolds, an initial set of 
design guidelines was distilled for software-based scaffolds. Two example 
overviews of findings are presented, followed by a description of some 
scaffolding design guidelines that were distilled from the full set of 
scaffold overviews. (Contains 33 references.) (AEF) 
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Introduction 



Many in education view a constructivist “learning by doing” approach as an effective way for 
novices to develop an understanding of new subjects and practices. This learning 
perspective is being adopted in many areas, such as science education, where students are 
encouraged to actively engage in complex science practices. However, science inquiry poses 
difficulties for novice learners. If learners are to engage in science inquiry, they need 
significant support, or scaffolding, to help them mindfully do the cognitive science tasks that 
are just out of their reach. One approach for supporting learners is to design computational 
tools that incorporate scaffolding features to make new practices accessible and visible so 
learners can engage in and understand these practices. In order to explore the systematic 
design and assessment of scaffolding features for software tools, we developed Symphony 
(figure 1), a comprehensive scaffolded work environment incorporating process scaffolding 
features for ninth-grade science students performing environmental science investigations 
(Quintana, 2001; Quintana, Eng, Carra, Wu, & Soloway, 1999). By analyzing how learners 
worked with the Symphony scaffolds, we distilled an initial set of design guidelines for 
software-based scaffolds. 




Figure 1: Symphony 



Background: Scaffolding for Learners Engaging in Complex Practices 

Current science education standards call for students to develop science process and content 
skills by actively engaging in scientific practices (American Association for the Advancement 
of Science, 1993; Davis, Hawley, McMullan, & Spilka, 1997; National Research Council, 
1996). Such practices involves substantive projects where students make observations; pose 
questions to study; gather and analyze data; propose explanations; and communicate their 
results to others. For example, in project-based science (Blumenfeld et al., 1991; Krajcik, 
Czerniak, & Berger, 1999), students develop science literacy by posing and investigating 
personally meaningful, open-ended driving questions, usually in pairs or groups, involving a 
variety of data collection, visualization, modeling, and argumentation activities over an 
extended time period (e.g., four to ten weeks). 

However, while different educational standards bodies state that students should actively 
perform substantive science investigations, the reality is that learners need significant support 
because of the complexity and open-ended nature of science inquiry (Krajcik et ah, 1998). 
For example, Quintana et ah (1999) note some overall roadblocks for learners who: 
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• Do not know what activities comprise a science investigation, or what activities to perform 
at different stages of the investigation. 

• Do not know how to do many of the science activities. 

• Lack the information that experts use when they investigate problems. 

In order to support learners, technology can incorporate scaffolding features that make 
complex practices accessible and mindfully doable by students. A scaffold can be described 
in the following manner: 

• A scaffold helps students do cognitive tasks that are just out of their current developmental 
and intellectual capability. From an instructional standpoint, scaffolds such as coaching, 
advising, critiquing, etc., are techniques that teachers use to make complex cognitive tasks 
accessible for students, (Collins, Brown, & Newman, 1989; Wood, Bruner, & Ross, 1975). 
From a technological standpoint, software scaffolds, such as templates and prompts for 
example, can be described as software features that help learners perform inaccessible 
cognitive activities. 

• A scaffold supports mindful work by learners. In other words, scaffolds should not make 
new activities too automatic or too easy to complete (Reiser, 2002). Just as good teachers 
guide students towards solutions without actually giving them the solution, scaffolds 
should guide and support learners, but in a way that learners still need to think about the 
work they are doing. 

• A scaffold “fades away” (or is removed from the software) when a learner no longer needs 
the given support (Jackson, Krajcik, & Soloway, 1998). Once learners have internalized 
the supported task, they no longer need the scaffold, which may now actually interfere 
with their work. 

Research Questions 

The Symphony project explored scaffolding issues by focusing on comprehensive process 
scaffolding in software tools for learners engaging in complex science practices requiring the 
coordinated use of those tools. For example, scientists use a range of tools to do their 
different scientific activities: tools for data collection, visualization, planning, modeling, 
argumentation, etc. We wanted to integrate such a tool set into a single scaffolded work 
environment for science inquiry. Our research questions concerned the design and 
assessment of process scaffolds supporting learners doing science inquiry. 

Scaffold Design Question 

We are interested in having students engage in complex scientific activities, something that 
is difficult for them to do. Thus we wanted to explore how to design scaffolds that support 
students in performing science investigations. Our design questions asked what conceptual 
process scaffolding strategies would support learners and how can we realize those 
scaffolding strategies in software? 

Scaffolding Assessment Question 

In terms of scaffolding assessment, we asked: to what extent are process scaffolds successful. 
How do learners do complex scientific activities with these scaffolds and how does their use 
of the scaffolds change over time. Scaffolding assessment differs from typical software 
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evaluation, where the main goal is to assess software usability (Mayhew, 1999). Scaffolding 
assessment is primarily concerned with seeing how well scaffolds support learners in doing 
and learning new practices. In other words, scaffolding assessment needs to consider the 
“effects with” scaffolding (i.e., how do learners use different scaffolds to perform the task 
supported by the scaffold?) and “effects of’ scaffolding (i.e., what do learners learn about a 
task after using the software for an extended time period?) (Salomon, Perkins, & Globerson, 
1991). 

Assessment methods for the “effects of’ technology are more straightforward and well- 
defined: pre- and post-testing, baseline assessments, and transfer task assessment to see what 
learners know about a work practice before and after using a scaffolded tool. However, 
there is less work on systematic assessment methods for “effects with” assessment. Thus the 
Symphony study focused on observing how learners worked with the different scaffolds over 
time to assess how the scaffolds supported learners in their science inquiry. 



Related Work 

Previous scaffolding research explored how scaffolding techniques used by teachers and 
mentors could be implemented in software and how scaffolds can fade from software 
(Guzdial, 1993; Metcalf, 1999). Other researchers are describing and evaluating different 
kinds of scaffolding strategies (Squires & Preece, 1999). Much scaffolding research revolves 
around the development of different tools for learners, especially science-oriented software 
to explore scaffolding in the context of science inquiry. For example, Model-It (Jackson et 
al., 1998; Metcalf, 1999) is a scaffolded tool supporting system dynamics modeling. Model- 
It not only explored the kinds of scaffolding needed to support system modeling, but also 
the ways in which the scaffolding can fade. Other scaffolded tools explored supporting the 
larger science inquiry process. The Knowledge Integration Environment (KIE) (Slotta & Linn, 
2000) and its successor, the Web-based Integrated Science Environment (WISE) helps students 
integrate evidence to develop and restructure their understanding of complex scientific 
ideas. The Biology Guided Inquiry Learning Environment (BGuILE) (Smith & Reiser, 1998; 
Tabak et ah, 1995) helps students engage in evolutionary biology investigations by implicitly 
incorporating domain-specific strategies and knowledge used by expert biologists. The 
Notebook-based Research and Investigative Support System (NoRIS) (Quintana, Abotel, & 
Soloway, 1996) was a precursor to Symphony exploring the design of scaffolded software 
tools for an intermediate to expert-level audience (e.g., undergraduates students to expert 
scientists) to support them in performing investigations in nuclear engineering. 

These tools all explored scaffolding in learner-centered scientific software for supporting 
different aspects of science inquiry. Our work in the Symphony project focused on 
comprehensive process scaffolding for the overall science inquiry process, on more 
systematic scaffolding design and assessment methods, and on articulating some scaffolding 
“lessons learned” from our observations. 
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Symphony: Designing and Assessing Process Scaffolds for 
Environmental Science Investigations 

In order to design the necessary scaffolds, we engaged in the learner-centered design (LCD) 
process we have developed to identify where students need support for their work 
(Quintana, Krajcik, & Soloway, 2001). Essentially, the LCD process involves: 

• Analyzing the practice (i.e., science inquiry) to fully identify its important aspects and 
components. 

• Identifying areas in the practice where learners may encounter difficulty and need 
support — we call these areas learner needs. 

• Determining the conceptual scaffolding strategies that address the different learner needs 
and the specific ways to realize those conceptual strategies in software. 

By analyzing scientific practices and our target audience, we saw general areas of complexity 
for learners and we identified corresponding learner needs to address with scaffolding (table 
1) (Quintana et al., 1999): 



Table 1: Complexity of Science Inquiry and Corresponding Learner Support Needs 



Characteristics 
of Science 
Inquiry 


Complexity for Learners 


Corresponding 
Learner Needs 


Complex 


Science inquiry involves completing many activities (e.g., 
planning, data collection, modeling, etc.), but there are no 
pre-defined activity sequences for solving a science 
problem. Different problem solvers perform different 
activities to investigate the same question. _ 


Learners need to see the range 
of possible inquiry activities 
and see how to perform those 
activities to progress through 
their work. 


Ill-Defined 


Science problems are ill-formulated and non-deterministic. 
Problem solvers explore and iterate through different 
alternatives in their investigation since there is no set 
linear path through the space of process activities. 


Learners need support to move 
among the inquiry process 
activities in a non-sequential, 
iterative manner. 


Opportunistic 


There are no explicit stopping rules defining when a science 
problem is "solved”. Accumulated results constantly define 
the direction of the investigation as problem solvers 
constantly review their progress and select activities they 
feel will bring them closer to an adequate answer. 


Learners need to understand 
the rationales for performing 
certain activities and the kinds 
of things experts reflect on to 
assess their results and drive 
their investigation. 



Symphony Process Scaffolds 

Unpacking these general learner needs helped us outline more specific learner needs to 
address with different scaffolding features. For each scaffold, we describe the specific work 
context for the scaffold, the learner need addressed by the scaffold, the conceptual 
scaffolding strategy for addressing the learner need, and the specific way we implemented 
the scaffold in Symphony. Here we present two scaffold examples to ground our discussion. 
Other Symphony scaffolds are found in Appendix 1 . 

Example Scaffold 1: Show Activity Options 

The “Show Activity Options” scaffold gives learners an overview of the activities that are 
part of their science inquiry. Table 2 summarizes the scaffold. 
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Table 2: Summary Description for “Show Activity Options” Scaffold 



Context 


Learner Need 


Scaffolding Strategy 


Science inquiry involves a 
range of different activities, 
but there are no specific, pre- 
determined activity sequences 
to solve a problem. Learners 
lack an expert's conceptual 
understanding of science 
inquiry and thus do not know 
what activities are involved 
throughout the inquiry process 
so they can proceed with their 
investigations. 


Therefore, learners need 
activity option 
information to see what 
inquiry activities and 
metaprocess activities 
are possible at different 
points in the 
investigation. 


Process maps provide activity option information 
with global views of activity spaces. Process 
maps show the possible work activities with a 
conceptual representation showing the structure 
and meaning of different activities (Kress & van 
Leeuwen, 1996). Other researchers say that 
visual constructs for "visualizing process” can be 
useful to help learners negotiate and internalize 
the process (Favorin & Kuutti, 1996; Feurzeig, 
1992). Therefore, process maps should describe 
the space of possible science activities that 
students can select from to help them proceed 
with their investigations. 



There are two implementations of the “Show Activity Options” scaffold in Symphony. The 
first is the main planning process map (or process wheel) in the planning workspace. The 
process wheel provides activity-option information describing the space of possible science 
inquiry activities that students can select from for their investigation plan (figure 2). 



Investigation Process 






i DevelopiPrtsbiem 






;-;/KV 

y y ? • V 7? 



. Reviow' Progress i 






V- 



V‘ (uyisuoiaeiData?^ .**♦/. 



Figure 2: Main Inquiry Process Wheel 



The second implementation of the “Show Activity Options” scaffold is the Conductor 
window (figure 3), which contains a metaprocess map describing the space of possible 
metaprocess activities to answer the student question “what can I do now?”. Where the 
main process wheel shows the inquiry activities that students can add to their plan, the 
Conductor window describes higher-level activities that students can perform, such as 
modifying the plan, performing an activity in the plan, etc. 




Figure 3: Metaprocess “Conductor” Window 
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Scaffold Example 2: Show Reflective Information 

The “Show Reflective Information” scaffold supports learners with reflection on important 
parts of their work. Table 3 summarizes the scaffold. 



Table 3: Summary Description for “Show Reflective Information” Scaffold 



Context 


Learner Need 


Scaffolding Strategy 


Experts need to reflect on various aspects 
of their investigation. However, reflection 
is an implicit activity. Learners may not 
realize the importance of reflection nor 
understand what they should reflect on as 
they engage in the science inquiry process. 


Therefore, learners need 
reflective information 
describing the information 
they should be thinking about 
when they perform different 
science inquiry activities. 


Explicit reflective information 
should describe what learners 
should be thinking about in their 
investigation and convey the types 
of information they need to 
articulate during their 
investigation. 



There are two implementations of the “Show Reflective Information” scaffold in 
Symphony. First, Symphony includes prompted activity templates (figure 4) in two reflective 
activity workspaces (i. e a “Develop Problem” workspace and Review Progress 
workspace) as seen in many software tools. Templates provide a notebook service so 
students can reflect on and articulate the information needed for these activities. Students 
can record necessary information in the template fields and the “title above each template 
field prompts the student about the information they need to think about. For example, the 
“Develop Problem” workspace has areas for students to reflect on their driving question, 
hypothesis, problem factors, plans, and notes (figure 4). 



The second implementation of the “Show Reflective Information” scaffold includes 
prompted activity text fields (figure 5). Similar to templates, prompted activity text fields are 
used to record information in different constructive activities (i.e., “Collect Data,” 

“Visualize Data,” and “Model Data”). There are two text fields in each constructive activity 
workspace (e.g., the “Visualize Data” workspace in figure 5). The “plan” text field is for 
planning the constructive activity (labeled “What do you hope to gain here? ) and the 
“learn” text field is for recording observations or other insights about the activity (labeled 
“What did you gain or learn here?”). Additionally, reflective questions are also found in the 
“plan” and “learn” text areas to prompt students about other important things to think 
about in these activities. 
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Figure 5: “Visualize Data” Activity Text Fields 



Details of the Research Study: Scaffolding Assessment Methods 

After designing and implementing the process scaffolds in Symphony, we assessed 
effectiveness of the scaffolds with a study where ninth-grade students used Symphony to 
perform air quality investigations. A science teacher in the high school where the study was 
conducted selected three pairs of students (five boys, one girl) to use Symphony for 
extended Michigan air quality investigations in a classroom setting. We focused on ninth- 
grade students because they had less experience in complex scientific investigations, being 
limited to smaller-scale, “canned” homework questions that involved the linear “scientific 
method” approach traditionally taught in school. Students engaged in a range of science 
inquiry activities using Symphony: planning, developing and setting up driving questions; 
collecting and visualizing data; building and analyzing models; and writing conclusions 
supported by evidence gathered in the previous activities. 

The study took place in an after-school environmental science class where students met for 
sixty to ninety minutes a day, four days a week. Students received some academic credit for 
the class and a small stipend. Students worked under the supervision of two people: the 
author of the study who served as a technical-expert (i.e., the “technical teacher”) and a 
retired scientist who served as a science process and content expert (i.e., the “primary 
teacher”). Essentially, the teachers adopted a “hands-off” without the information given (or 
WIG) approach (Hogan, Nastasi, & Pressley, 2000) where teachers do not “provide direct 
instruction or other exposure to the conceptual information”, instead guiding the students’ 
experience in productive directions without direcdy giving students the “right” answers. 
The technical teacher observed the students as they worked, managed the videotaping 
equipment, provided technical assistance when issues arose with the software, and assisted 
the primary teacher as needed. 

Students investigated driving questions about Michigan air quality organized in three 
different units. The primary teacher and a high-school science teacher wrote the questions. 
Additionally, students were able to propose their own questions for investigation. Table 4 
summarizes the problem units. 
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Table 4: Summary of Student Problem Units 



Problem Unit 


Example Questions 


Notes 


Unit One 
Three teacher- 
written and one 
student written 
questions 


• Does air quality (as determined by the 
EPA "dirty half-dozen” pollutants) stay 
the same or does it vary from day-to- 
day? 

• Scenario: You need to shut down air 
quality monitoring equipment for 
maintenance. What you recommend 
as the best time to shut down the 
equipment? 

• Is there a regular pattern of change for 
carbon monoxide? If so, what is it? If 
not, why is this the case? 

• Student-written question 


These exercises gave students a chance to 
explore the contents of the EPA pollution data in 
the database we constructed for their 
investigations. Students explored data for six 
pollutants (C0 2 , S0 2 , N0 2 , 0 3 , PM 10 , Pb) to see if 
there were patterns in pollution levels, similar 
patterns of change for the different pollutants, 
etc. After they explored the pollution data, 
students then posed their own question to 
investigate about pollution levels. 


Unit Two 
One teacher- 
written 
question and 
one student- 
written 
question 


• Develop a hypothesis about the 
relationship between nitrogen dioxide 
pollution and wind. 

• Student-written question 


These exercises gave students a chance to 
explore the relationships between pollution data 
and data about other factors that affect air 
quality, such as weather data. Again, after 
exploring some possible relationships, students 
posed their own question to investigate. 


Unit Three 
One teacher- 
written 
question 
leading to 
independent 
student 
investigation 


• Given various cities in Michigan, rank 
the cities in terms in terms of ozone 
pollution, starting with the city with 
the worst ozone pollution. 

* Independent student investigation 


Students were given a question with a seemingly 
obvious conclusion (e.g., Detroit has worse ozone 
than Manistee National Forest) contradicted by 
the actual data (e.g., Detroit has less ozone than 
Manistee). Students had to then conduct 
investigations using the skills and information 
gained in the previous units to explain the 
apparent contradiction. 



The units were designed to be open-ended with no single distinct “right” answers and with 
several investigative paths to pursue. Students had to investigate the questions and write 
conclusions supported with the work from their investigation. The teachers would review 
the students’ conclusions to see if they were reasonably supported. If their conclusions were 
weak, the students needed to refine their investigation. When the teachers were satisfied 
with the conclusions, students could proceed to the next question. 

Students worked in pairs on their investigations and were videotaped for the duration of the 
study. There were three stations in the classroom where students worked — one station per 
student pair — consisting of computers connected to process video kits that videotaped the 
computer screen as the students worked, but not the students themselves. The students 
wore clip-on microphones connected to the process video kits to record their commentary. 
Thus the primary data for the study was a videotaped record of the students’ Symphony 
work synchronized with their conversations. Students were also interviewed to get their 
impressions about the various Symphony features. 

Assessing the “Effects With” Scaffolding 

We observed how the student pairs worked with the different scaffolding features in 
Symphony over an extended time period. The core of the analysis involves a method 
described by Chi (1997) to iteratively decompose the video data into smaller units of activity 
that can be interpreted in a more structured manner. Specifically, the scaffolding 
assessment involved the following broad steps: 
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• Identifying specific episodes where students used scaffolds to perform different inquiry 
activities. 

• Assessing how students used different scaffolds in each episode using a new set of 
assessment criteria. 

• Summarizing the assessment information for each scaffold to describe how students used 
the different scaffolds over time. 

Assessment Phase One: Coding by Individual Inquiry Activity 
We began the assessment by reviewing the student video to identify and code specific 
episodes corresponding to a major science inquiry activity (table 5). We recorded 
information about each individual episode into database records describing: 

• Researcher observations on the practices students performed in the episode. 

• Instances where students needed help with the activities. These observations point out 
where students need additional or better-designed scaffolds. 



Table 5: Episodes for the Symphony Video Analysis 



Episodes 


Episode Description 


Planning 


Students determine what activities to select and add to their plan, select activities from their 
plan to perform, or refine their plan. 


Develop 

Problem 


Students work on developing their driving question, hypothesis, factors, specific plan, and. 
research notes. 


Data Search 


Students collect data needed for their investigation. 


Build Graphs 


Students build graphs to visualize data they have collected. 


Analyze 

Graphs 


Students examine and analyze graphs they have built. 


Build Models 


Students build system dynamics models to describe the behavior of certain systems. 


Analyze 

Models 


Students run and test the system dynamics models that they have built. 


Review 

Progress 


Students review the work they have performed, determine if additional work is necessary, and 
expand on their conclusion. 


Incompletion 


Students worked on an inquiry activity described above, but did not complete it. 


Technical 

Problem 


Students had a technical problem with the software, computer, or computer network. 



Assessment Phase Two: Scaffolding Usage Assessment 

After identifying each scaffold students used in their episodes, we needed to see how students 
used the scaffolds to do the work in each episode. Rather than simply formulating a binary 
“good” or “bad” rating for each scaffolding feature, we found that we needed to assess the 
scaffolds along different dimensions to discover finer shades of assessment information from 
both user-centered tool usability and science inquiry practice perspectives. Therefore we 
devised a set of criteria to address these considerations (table 6) (Quintana, Fretz, Krajcik, & 
Soloway, 2000). 

Scaffolds are supposed to support the mindful doing of complex tasks, thus the criteria 
consider how the students did their individual tasks with the support of the given scaffolds. 
The meaning of the different criteria with respect to a given scaffold must be defined for 
each scaffold, and in some instances, a criterion may not apply. However, by using a range 
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of evaluation criteria we could get more detailed assessment information to see the trade- 
offs for each scaffold. 



Table 6: Scaffolding Assessment Criteria 



Name 


Definition 


Accessibility 


Measures the initial usability of a scaffolding feature to describe whether can access or use the 
feature. Involves a binary "yes/no” answer: learners can either access the scaffold or they 
cannot. 


Use 


Measures whether learners use or ignore a scaffold that is obviously accessible to them. 
Involves a binary "yes/no” answer: learners either use or do not use an accessible scaffold. 


Efficiency 


Measures the speed or difficulty of performance when students use scaffolding features to do 
tasks. This should describe how fast or how easily learners can use a particular scaffolding 
feature to perform the supported task and how their task performance changes over time. 


Accuracy 


Measures whether a scaffolding feature supports the correct and appropriate completion of the 
supported task and how task accuracy improves over time. 


Progression 


Measures how learners progress through their tasks while using a scaffolding feature: are they 
working in a linear manner or a nonlinear manner? Learners will initially work in a linear, step- 
by-step manner. As learners develop expertise in a work task, their work should become more 
opportunistic, and nonlinear (Schoenfeld, 1987). 


Reflectiveness 


Measures the amount of task reflection learners engage in when using a scaffold to see how 
focused they are on their tasks and how their reflection varies over time. This shows whether 
additional support for "mindful” work is necessary (Salomon et al., 1991). 



For each student pair, we reviewed and assessed using the criteria every instance when they 
used a scaffold. This resulted in three sets of individual scaffold assessments per scaffold— -one 
set for each student pair. Next, we summarized every scaffold’s individual assessments in 
each problem unit to create one scaffold usage summary (or SUS) per scaffold for that 
problem unit. A scaffold usage summary describes how each student pair used the scaffold 
over the course of each problem unit. Since students did three problem units, every scaffold 
had three scaffold usage summaries per student pair. 

Assessment Phase Three: Scaffolding Meta-Summarization 

Our final assessment phase involved summarizing the scaffold usage summaries from phase 
two (i.e., creating metasummaries) to describe how scaffold usage changed over time and how 
different student pairs used scaffolds throughout the study. For every scaffold, we created 
two metasummaries: a group summary and a unit summary. We created group summaries by 
consolidating each student pair’s scaffold usage summaries for every scaffold. Thus each 
scaffold had one group summary per student pair describing how the student pair’s scaffold 
use changed over the course of the study. After the group summaries for the three student 
pairs were complete, we consolidated the group summaries for each scaffold into an overall 
group summary describing how all the student pairs used an individual scaffold over time. 
Thus each scaffold will have one overall group summary. 

Similarly, we created unit summaries by consolidating all the student pair’s scaffold usage 
summaries in every problem unit for each scaffold. While group summaries consider how 
an individual student pair used a scaffold over time, a unit summary describes how all the 
students pairs used a scaffold in a given problem unit. Thus each scaffold will have one unit 
summary per problem unit. After all the unit summaries for each problem unit were 
complete, we consolidated the unit summaries into an overall unit summary describing how 
the student pairs used a scaffold over time. Thus each scaffold will have one overall unit 
summary. Note that a scaffold’s overall group and overall unit summaries should closely 
match to verify each other. 




1 0 
A 
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Finally, we reviewed each scaffold’s overall group and unit summaries to write a final 
assessment overview describing the “lessons learned” from the scaffold assessment. The 
assessment overview should discuss the pros and cons of the scaffold in the given work 
context and how the scaffold could be redesigned and improved. 

Figure 6 describes the scaffold summaries created in this assessment phase. Figure 6 
describes the assessment of a general “scaffold x” since we follow this process for every 
scaffold in the software 



Scaffold Assessment 




Figure 6: General Overview of Scaffold Usage Summaries 

Research Findings from the “Effects With” 

Scaffolding Assessment 

After following the procedure summarized in figure 6, we had an assessment overview for 
each scaffold in the software. While we cannot fully describe all the assessment information 
for the entire set of scaffolds (see Quintana (2001) for the full results), below we present two 
example overviews below. After presenting the example findings, we describe some 
scaffolding design guidelines that we distilled given the full set of scaffold overviews. 
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Example Findings: The Process Wheel 

The process wheel supported activity selection by visualizing the space of possible inquiry 
activities students could select for their investigation plan. The process wheel was accessible 
since it is the central interface element on the planning workspace. All the students used the 
process wheel since they could not continue with their investigation until they selected 
activities for their plan. Students used the process wheel efficiently, selecting activities for 
their plan without difficulty. So from a user-centered standpoint, the process wheel was 
usable for the students. 

In terms of the more learner-centered criteria, we saw that accuracy and progression were 
connected because linear progression (i.e., occasions when learners selected activities in the 
clockwise “wheel order” implied by the wheel) led to lower accuracy. Most students initially 
selected all the process wheel activities in the implied clockwise “wheel order” without 
considering whether they actually needed those activities in their plan. As a result, the 
students selected extraneous activities (e.g., adding modeling to the plan even though 
modeling was not needed in a problem), thus lowering their accuracy rating. Over time, 
accuracy increased and progression became more nonlinear. At that point, students selected 
appropriate activities as needed without strictly following the wheel order, thus adding 
fewer extraneous activities to their plan. 

Reflection was more visible in earlier problem units, when there was more discussion between 
students about the activities to select. As students proceeded through the different problem 
units, there was less discussion, but since students quickly selected accurate activities with 
nonlinear progression, implying that they better understood the purpose of the different 
inquiry activities. 

The assessment results suggest some design implications for process maps. A visual 
representation like the process wheel can help steer students into thinking about what 
activities to perform before they rush to perform them, something that students are prone 
to do (Collins, 1996). Of course, the risk of a visual representation like the process wheel is 
that students will simply follow the order implied in the wheel. In that case, the wheel 
becomes less of a support and more of a crutch, where some students are simply following 
the wheel order rather than making mindful selections. 

Designers certainly need methods to visualize activity spaces for learners facing a new work 
practice. But an effective visualization should force learners to be mindful about their 
activity selection. Designers must be wary of the implications that a visual representation 
brings (e.g., the implied clockwise order in a wheel). Certainly, complex work practices like 
science inquiry can be partially ordered (e.g., one cannot graph data before collecting data) 
and this partial order could be visually represented to help learners see “appropriate” 
selections. However, designers should be cautious in the choice of representation so 
learners will not blindly follow any implied order in the representation, resulting in less- 
mindful and inappropriate activity selection. 

Example Findings: Activity Templates and Text Fields 

The activity templates and text fields supported learner reflection and information 
recording throughout an investigation. Students used the activity templates fairly well. The 
templates were efficient to use and accessible since the reflective activity workspaces consist 
solely of the templates. 




Quintana/AERA 2002/March 7, 2002 



14 



Page 13 of 25 



* 



Reflectiveness ratings for both activity templates were good as we saw many instances when 
students discussed the information they had to record. In those instances when students had 
to work alone, they still showed explicit signs of reflection: making audible comments about 
their work (e.g., commenting that their hypothesis is wrong or commenting about new tasks 
to perform), discussing their work with teachers, or reviewing prior work as they recorded 
information in the template. 

Accuracy ratings were mixed because while students wrote mostly appropriate information 
when they used fields in the template, they used different template fields to varying degrees. 
We also saw instances when students would verbalize important information (raising the 
reflectiveness rating) without recording the information in the template (lowering the accuracy 
rating because recording information was part of the task). For example, students would 
comment on new activities to perform, but they would not modify their plan to record those 
changes. It was important that the students were thinking about their science activities. But 
the fact that students would verbalize information without recording implies that more 
support is needed in the template so students can keep an explicit work record. 

One drawback of a template is that it is simply a collection of text fields and learners might 
see the fields as being optional, especially if they are not clear about the information they 
should be recording. One approach for addressing this might be to use a “forcing” 
mechanism in the template so that learners cannot continue with other work until they have 
entered appropriate information in all the template fields. Another design consideration 
would be to allow other media types and other modes of information entry, such as voice 
entry. Since students did discuss important observations without recording them in the 
template, perhaps a mechanism allowing learners to talk to the system and leave quick 
“voice notes” would allow them to quickly leave an explicit record of their actions. 

Unlike the full activity templates, we found that students rarely used the “plan” and “learn” 
activity text fields in the constructive workspaces. One reason could revolve around the 
perceived student focus of the activity workspaces. The reflective activity workspaces (i.e., 
“Develop Problem,” “Review Progress”) consist solely of the template, so the activity’s focus 
is on reflection and recording information. However, the constructive activity workspaces 
contain a variety of elements (see “Visualize Data” workspace in figure 5): the activity text 
fields, a process map to help construct the relevant artifact for the activity (e.g., a graph), 
and artifact lists. So students saw artifact construction as the primary focus of the workspace 
and instead felt that recording plans and observations were more ancillary activities. 

The activity text fields are straightforward to use, but they need to be made more explicit in 
the activity workspaces. One approach to emphasizing the importance of reflecting on plans 
and observations would be to have the system force the students to write planning 
information before artifact construction. For example, students could not build graphs until 
they recorded some plans for their graphing activity. Furthermore, after building some 
graphs, students would not be able to proceed with another inquiry activity until they 
recorded some observations so that they would be forced to slow down and think about the 
artifacts they are building. However, it may be difficult to implement a system that 
effectively determines when the learner has written “appropriate” information. 
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Another approach might be to allow learners to connect information to the different work 
activities they are performing. For example, if students entered a specific plan earlier in the 
work process, they might simply stop and connect that portion of their plan to the current 
activity. If students later record observations about some artifact like a graph, they should 
be able to connect their observations back to the artifact itself. Some argumentation tools, 
such as Belvedere (Suthers, Weiner, Connelly, & Paolucci, 1995) attempt to support 
students in building an argument by visually connecting hypotheses with evidence. A 
similar approach can be used so learners can create and connect plans, artifacts, and 
conclusions to see the importance of reflecting on artifact construction plans and artifact 
analysis. 

Interpretation: Scaffolding Design Guidelines 

After completing the scaffold overviews, we reviewed and s>mthesized the assessment 
information to determine some initial scaffolding design guidelines explaining why students 
used the different scaffolds in the manner that they did. We distilled these guidelines by 
isolating different scaffolding characteristics to see if there were similar scaffold use patterns 
correlated with the scaffolding characteristics. We looked at the following scaffold 
characteristics: 

• Visibility of the scaffold in its workspace 

• Essentialness of a scaffold’s use in terms of doing the target work 

• Coupling of a scaffold with other scaffolds in a workspace 

• Usability of a scaffold 

• Representation of a scaffold in terms of its graphical or textual nature 

The following sections describe each of these scaffold characteristics and the associated 
scaffold design guidelines describing how that characteristic may have impacted scaffold use 
by learners. (A summary table of the Symphony scaffolds and their associated 
characteristics is found at the end of appendix 1 .) 

Scaffold Visibility . 

Visibility describes how visible a scaffold is in the workspace where it is located. 1 able 7 

shows the two descriptions of scaffold visibility. 



Table 7: Definition of Scaffold Visibility 



Scaffold 

Visibility 


Description 


Visible 


The scaffold is always visible in the workspace where it is located. 


Triggered 


The scaffold must be triggered by some control in the workspace or triggered by the system. 



Visibility is important because our findings showed that visible scaffolds were used more 
than triggered scaffolds. Most of the triggered scaffolds conveyed explanatory information 
(e.g., information that explained some science inquiry concept). However, even though 
students showed the need for such information (e.g., students had questions about factors or 
hypotheses), they asked teachers or other students for this information instead of triggering 
the scaffolds. Thus the first design guideline suggested by these findings is: 
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Guideline 1: Scaffold Visibility. Scaffolds and scaffold content should be 
made as visible as possible because learners may not scaffolds. Rather than 
having learners trigger a scaffold, scaffolds should be initially visible and 
learners can remove them when no longer needed. 

This guideline suggests that designers should not assume that learners would trigger 
scaffolds. Triggered scaffolds can be problematic because some learners may not want to 
admit that they need help and will not trigger the scaffold. Even scaffolds triggered by the 
system tend to be ignored. For example, Model-It triggers messages to students when they 
have not performed important parts of the system modeling task. However, students tended 
to ignore these supportive scaffolds (Metcalf, 1999). 

Avoiding triggered scaffolds in favor of visible scaffolds runs counter to the traditional user 
interface approach for designing help features. Typically in user interface design, users 
trigger online help features when they need help. However, the Symphony findings suggest 
that the approach for LCD scaffolds should be reversed. Rather than turning scaffolds on 
when they need help, scaffolds should be initially visible when a given workspace opens. 
Learners can then turn the scaffolds off when the help is no longer needed. 

Scaffold Essentialness 

Essentialness describes how essential it is for learners to use a scaffold to proceed with their 
work tasks. Table 8 shows the three descriptions of essentialness. 



Table 8: Definition of Scaffold Essentialness 



Scaffold 

Essentialness 


Description 


Essential 


Use of a scaffold is mandatory for learners to perform a work task so they can proceed with 
their work. 


Semi-Essential 


Learners can use a scaffold to do a given work task so they can proceed with their work, 
but there are other scaffolds that can be used to perform the same task. 


Optional 


Learners can choose to use a scaffold to perform a given work task, but performance of the 
task is not mandatory (although it would probably be helpful). 



The Symphony findings showed that learners tended to bypass the optional scaffolds and 
use the essential and semi-essential scaffolds. This is obvious for the essential scaffolds, 
since learners need to use those scaffolds to proceed with their work. But all the optional 
scaffolds were bypassed except for the prompted activity templates. Thus the second design 
guideline suggested by these findings is: 

Guideline 2: Scaffold Essentialness. Scaffolds should be designed to have 
an essential character. There should be few optional scaffolds because 
learners may bypass them and miss the support needed to perform certain 
work tasks. 

Note that the lack of scaffold essentialness on its own does not necessarily mean scaffolds 
will be ignored. For example, the prompted activity templates were optional, but still used 
well. However, the findings suggest that if scaffolds are optional, there needs to be a way of 
focusing the learner’s attention on the scaffold to emphasize the scaffold’s importance. For 
example, optional scaffolds could either have enticing triggers to draw the learner’s 
attention or be essential so learners will have to use the scaffold to proceed with their work. 



Quintana/AERA 2002/March 7, 2002 



17 



Page 16 of 25 



# 



Having more essential scaffolds can force learners to engage in important parts of the work 
process. The trade-off for designers involves incorporating essential scaffolds that do not 
result in frustrating software. If the collection of essential scaffolds requires learners to 
work in a strictly regimented manner, learners may become frustrated and lose interest in 
the work. One approach could be to use essential scaffolds for learners who are starting out 
with the software. These learners would ostensibly be new to the work, so forcing scaffold 
use would be advantageous to expose them to the different work tasks. When learners 
develop a better internal work model and become more opportunistic in their work style, 
the essential scaffolds may begin to interfere with their work. At that point, learners could 
fade scaffolds by “reducing” their essentialness (i.e., making the scaffolds optional). Thus, 
scaffold essentialness seems closely tied to the fading issues studied by Metcalf (1999). 

Scaffold Coupling 

Coupling describes how a scaffold is connected or related to other scaffolds in the 
workspace where they appear. Table 9 shows the three descriptions of coupling. 



Table 9: Definition of Scaffold Coupling 



Scaffold 

Coupling 


Descriptions 


Tightly Coupled 


The scaffold is connected to other scaffolds on a workplace such that the scaffolds work in 
concert or an action a scaffold visibly affects the connected scaffolds. 


Loosely 

Coupled 


The scaffold appears with other scaffolds on a given workspace, but the scaffolds are no ( t 
connected in any manner. 


No Coupling 


The scaffold appears alone on a given workspace. 



Coupling effects were seen in the Symphony constructive workspaces (e.g., figure 5), which 
consist of the loosely coupled prompted activity text fields (for reflection) and process maps 
(for constructing data sets). None of these scaffolds are directly dependent on the others, 
nor do the actions on one scaffold affect the others. The findings showed that learners used 
the process maps but bypassed the activity text fields. This is not surprising, especially when 
looking at the essentialness measure: the process maps were essential for building a graph 
but the activity text fields were optional. Since students can proceed with their constructive 
activities without taking notes, students instead focused on the more essential scaffolds, 
bypassing the text fields. So the third design guideline suggested by these findings is: 

Guideline 3: Scaffold Coupling. Scaffolds that appear together in a 
workspace should be as tightly coupled as possible so that learners will focus 
their attention on all the scaffolds. Loosely coupled scaffolds risk being 
bypassed by learners if their focus is only directed to some subset of scaffolds 
on the workspace. 

Coupling is important because it is one way of directing the learner’s focus on the scaffolds 
so they will perform the various work tasks. Scaffolds that appear alone will certainly get 
the learner’s focus. For example, the prompted activity templates, which were not coupled 
since they appeared alone on a workspace, were used well despite being optional. If there 
are multiple scaffolds on a workspace, tightly coupling them will help direct the learner’s 
focus. Since scaffolds must be used together, or since action on one scaffold results in a 
visible effect on another scaffold, the learner is implicitly drawn to all the scaffolds. If a 
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workspace does contain multiple, loosely coupled scaffolds, then those scaffolds might be 
made essential so that learners will not bypass them. 

Scaffold Usability 

Usability simply describes whether there were any usability problems with the scaffolds. 
Table 10 shows the three descriptions of usability. 



Table 10: Definition of Scaffold Usability 



Scaffold 

Usability 


Description 


Detrimental 


There were enough usability problems to interfere with appropriate use of the scaffold so 
that the learner could not or did not do the underlying task. 


Problematic 


There were usability problems with the scaffold, but the learner was still able to perform 
some or most of the underlying work task. ____ 


None 


There were no major usability problems with the scaffold. 



Usability can definitely affect how learners use scaffolds. In the Symphony study, there 
were three major categories of scaffold usability problems: 

• Problems with scaffold “triggers.” There were some examples of this problem, such as 
scaffold buttons that did not appear to be a clickable control to some students, or scaffold 
buttons with generic labels that did not specifically convey the information provided in the 
scaffold. 

• Problems with non-obvious scaffold functionality. One scaffold was functionally 
overloaded so that some functionality was obvious, but other functionality was not. 

• Problems with confusing scaffold functionality. One scaffold had a similar graphical 
layout to another, but different behavior. 

Thus usability issues did affect the use of the scaffolds. So the fourth design guideline 
suggested by these findings is: 

Guideline 4: Scaffold Usability. Scaffolds need to be usable or learners will 
not be able to mindfully perform the supported tasks. However, scaffolds 
should not be designed to make new tasks too automatic to complete. 

Usability problems can be frustrating and can cause learners to bypass scaffolds, thus 
missing important work activity. So user-centered design principles need to be considered 
for scaffold design to ensure that usability issues do not interfere with scaffold use. 

However, there is a trade-off between “just enough” and “too much” usability. Since ease- 
of-use is not the primary concern for learner-centered design, some user-centered design 
principles may need to be disregarded in scaffold design to avoid making work tasks too 
easy, thus discouraging the “mindful work” goal for LCD. This trade-off between user- 
centered goals (i.e., tool efficiency) and learner-centered goals (i.e., mindful work) is an 
ongoing research topic (e.g., Rappin, Guzdial, Realff, & Ludovice (1997); Sedighian, Klawe, 
& Westrom (2001)). 
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Scaffold Representation 

Representation simply describes the visual nature of the scaffold. I able 1 1 shows two 
descriptions of representation. 



Table 11: Definition of Scaffold Representation 



Scaffold Representation 


Description 


Graphical 


The scaffold is primarily graphical in nature. 


L_ 

Textual 


The scaffold is primarily textual in nature. 



Our findings showed no direct correlation between the scaffold s visual representation and 
how the scaffold was used. So the fifth design guideline suggested by these findings is: 



Guideline 5: Scaffold Representation. Neither textual or graphical scaffold 
representations are inherently better or worse than the other. Textual 
scaffolds can be simple, but useful for learners. Graphical scaffolds can be 
powerful and motivating, but misleading to learners. 

Intuitively, it would seem that graphical scaffolds would be more motivating and effective 
for learners than textual scaffolds. However, there was no evidence that graphical scaffolds 
were inherently more effective than textual scaffolds. Some textual scaffolds were used well 
(e.g., prompted activity templates), while some were not (e.g., prompted activity text fields). 
So scaffolds were not used or bypassed simply because of their textual nature. 

Conversely, graphical scaffolds can be effective, but they must be designed with caution so 
that they are not misleading to learners. Designers need to be wary of information that 
learners might interpret from a visual representation. For example, the Symphony process 
wheel is useful for conveying the space of inquiry activities in the science inquiry process. 
However, the design of the process wheel was somewhat misleading because the wheel 
representation conveyed an unintended activity selection order during planning. 

Concluding Remarks and Future Directions 

In classroom trials, students effectively used Symphony to engage in extended air quality 
investigations. By looking at how students used the individual Symphony scaffolds during 
their investigations, we gained information describing the strengths and weaknesses of 
different process scaffolds and how different features of scaffolds affected the learners 
scaffold use. The scaffolding assessment information led to a set of initial scaffolding design 
guidelines for designers implementing scaffolding features to support novice learners in 
doing complex cognitive tasks. The scaffolding design guidelines give some insight into 
different properties of software features and how those features should be defined to 
support learners. However, it is still incumbent on designers to consider the information in 
the full guideline set to understand the interplay between the different scaffold 
characteristics and the inevitable design trade-offs that are found in any design activity. Our 
future work involves using our LCD methods to continue designing scaffolds for new 
complex tasks and assessing the scaffolds with a range of learners (e.g., K-12 students) on a 
range of devices (e.g., desktop to handheld computers) to test, refine, and expand our initial 
set of scaffolding design guidelines to provide design guidance for designers of learner- 
centered scaffolded tools. 
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Appendix 1 



The following describes some of the other Symphony scaffolds in the classroom study. 

Scaffold Name: Show Activity Procedures 



Table 12: Summary Description for “Show Activity Procedures” Scaffold 



Context. 


Learner Need 


Scaffolding Strategy 


Many of the activities in the 
science inquiry process, especially 
constructive activities, are 
complex and require procedures 
that learners do not know how to 
perform. 


Therefore, learners need procedural 
information to see how to perform 
the constructive activities, such as 
data collection, data visualization, 
and modeling. 


Just as process maps visualize 
activity spaces, process maps can 
similarly incorporate process 
visualization techniques to convey 
the procedures needed to 
complete certain inquiry activities. 



Symphony implements the “Show Activity Procedure” scaffold by using flow diagrams. A 
flow diagram is a task-oriented directed map displaying the procedure for constructing 
different artifacts, such as a graph (figure 7). Similar flow diagrams are used to help 
students build data sets and models. A flow diagram is a classtficational diagram, which 
describes an active process and shows the interconnections between the parts of the process 
(Kress & van Leeuwen, 1996). Pressing a button in the flow diagram triggers the 
appropriate portion of the tool used to complete the selected part of the artifact 
construction task. 




Figure 7: “Visualize Data” Flow Diagram for Building Graphs 



Scaffold Name: Show Explanatory Information 



Table 13: Summary Description for “Show Explanatory Information” Scaffold 



Context 


Learner Need 


Scaffolding Strategy 


Experts use different kinds 
concepts, rationales, and 
terminology about a variety of 
work aspects to perform 
different activities. However, 
learners lack or have incomplete 
knowledge of these work 
concepts and activities. 


Therefore learners need 
explanatory information that 
describes different concepts and 
terms from the work practice 
(e.g., What is a "hypothesis”?) 
along with the purposes of the 
different inquiry activities so they 
can proceed with their work. 


Different guides can explain important 
concepts learners need to complete 
their tasks within the appropriate 
activity workspaces. The guides are a 
form of work-oriented help that 
support learners with some aspect of 
the underlying work practice rather 
than helping with tool operation (the 
traditional help provided in software). 
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Symphony implements the “Show Explanation Information scaffold in two ways. First, 
Symphony implements rollover activity rationale guides (figure 8). Process wheel activities 
and flow map tasks have a connected activity rationale guide describing the given task or 
activity. These guides appear when the mouse pointer is positioned over the activity or task 

buttons. 

Second, Symphony implements “Info” guides to explain concepts (e.g.> driving question, 
hypothesis, factors, general plan, research notes) in the “Develop Problem workspace 
(figure 9). A guide for a particular concept is triggered from an “Info” button next to each 
template field. 




Figure 8: Rollover Activity Guide Example 




Step 1 I ; 

Activity: Develop Problem 






R Info: Driving Ouettjotwjgg 






k OTYinq question ts a question tnat you want to ask-someming mat you want 
o find me answto. The driving quesoon should be meanintful to you and t 
ihouW atso ce something that you can research (and hopefully find an answer 
s) wthin the given tme frame. 


- 




• . \;y 

■ - - ■ •«♦• ■ . • 




Some students driving questions related to ozone are: 

* Are the ozone levels in Ann Aiborme same or afferent on football Saturdays 
as compared to non-football Saturdays? 




: Write your ge nera ip ta n :|['(info 


g; 










* How do the ozone levels compare for a rural town, Ike Dexter, and an urban 
center, Ike Detroit?! 




! 























Write your hypothec i s: , j ln?o ip- 




Figure 9: “Info” Guide Example 



Quintana/AERA 2002/March 7, 2002 



22 



Page 21 of 25 





Scaffold Name: Allow Flexible Planning 



Table 14: Summary Description for “Allow Flexible Planning” Scaffold 



Context 


Learner Need 


Scaffolding Strategy 


Because of the chaotic, iterative nature of 
the science inquiry process, flexible 
planning is important to refine the 
direction of the investigation as new 
results warrant. Scientists constantly 
modify their existing plans by rearranging 
activity order, deleting activities, or 
adding new activities. But learners may 
not be fully aware of the implicit planning 
task and thus lack experience in planning 
and managing the complex set of 
activities needed to engage in the work. 


Therefore learners need 
explicit support for the 
implicit planning activity that 
experts perform to refine and 
maintain a record of the 
investigation. 


Explicit flexible planning services 
are needed to give learners a 
mechanism for incrementally 
planning their investigation. 
Explicit planning services for 
learners can help them see the 
importance of constant planning 
in an iterative, chaotic work 
practice. 



Symphony implements the “Allow Flexible Planning” scaffold with a plan/log grid (figure 1 0) 
for plan setup and modification. Students set up plans by taking activities from the process 
wheel and adding them to the plan row of the grid. Students modify their plans by moving 
activities to different locations in the grid or by removing activities from the grid. By 
serving as a flexible time line, the plan grid provides a temporal analytical diagram (Kress & 
van Leeuwen, 1996) that relates a part- whole structure (i.e., the overall plan) as successive 
stages (i.e., the steps in the plan) to support flexible planning and re-planning. 




Scaffold Name: Support Non-Sequential Activity Traversal 



Table 15: Summary Description for “Support Non-Sequential Activity Traversal” Scaffold 



Context 


Learner Need 


Scaffolding Strategy 


Since science inquiry is 
opportunistic and chaotic, the 
work involves moving between the 
different process activities (e.g., 
moving from planning to data 
collection to visualization to 
planning, etc.) in a non-sequential 
manner. Experts can work in a 
chaotic manner unlike learners 
who are more familiar with linear 
work. 


Therefore learners need support to 
facilitate non-sequential movement 
between the different activities in 
the science inquiry process so that 
they can work in an opportunistic 
manner. 


A SWEet should allow flexible 
activity traversal with mechanisms 
that help learners quickly traverse 
the different activity work spaces 
as needed throughout the 
investigation rather than forcing a 
linear activity sequence. 
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Symphony implements the “Support Non-Sequential Activity Traversal” scaffold in two 
ways. First, Symphony uses tabbed activity workspaces (figure 8) to facilitate rapid movement 
between activities. Symphony allows students to open several activity workspaces 
simultaneously and move quickly through them (via tabbing) in the iterative, non-linear 
manner indicative of realistic science work. Second, Symphony uses the log row buttons in 
the Plan/Log grid (figure 6) to display activities that learners have already worked on. 
Learners can open previous activity workspaces by pressing the log row button 
corresponding to the activity. 

Table 16 summarizes the different Symphony scaffold characteristics and general findings. 



Table 16: Scaffold Characteristics and General Findings 



Scaffold 


Visibility 


Essentialness 


Coupling 


Usability . 


Representation 


General 

Findings 


Process wheel 


Visible 


Essential 


Tightly coupled 
with the 
plan/log grid 


None 


Graphical 

(Map) 


Useful, albeit 
somewhat 
misleading for 
activity 
selection. 


Plan/ log grid 


Visible 


Essential 


Tightly coupled 
with the 
process wheel 


Problematic: 
Overloaded 
functionality results 
in non-obvious plan 
"modify” 
functionality 


Graphical 

(Grid) 


Used to set up 
plans, but not 
to modify 
plans. 


Prompted 

activity 

templates 


Visible 


Optional 


None 


None 


Textual 


Used well to 
record 

information for 

reflective 

activities. 


Flow maps 


Visible 


Essential 


Loosely coupled 
with activity 
text fields and 
artifact lists 


None 


Graphical 

(Map) 


Used to access 
artifact 
construction 
services. 


Tabbed activity 
workspaces 


Visible 


Semi*essential 


Tightly coupled 
to activity 
workspaces 


Sparse labels could 
be confusing 


Graphical 

(Tabs) 


Used with 
increasing 
frequency. 


Log row buttons 


Visible 


Semi-essential 


Tightly coupled 
to plan /log grid 


None 


Graphical 
(Push buttons) 


Used with 

decreasing 

frequency. 


Workspace 
artifact lists 


Visible 


Semi-essential 


Loosely coupled 
with activity 
text fields and 
flow maps 


None 


Textual 


Used well to 

access 

artifacts. 


Prompted 
activity text 
fields 


Visible 


Optional 


Loosely coupled 
with flow maps 
and artifact 
lists 


None 


Textual 


Used the 
"learn” fields 
sparingly. Did 
not use the 
"plan” fields. 


Artifact table* of* 
contents 


Triggered 


Semi-essential 


None 


Problematic: Trigger 
button could be 
more visible 


Textual 
(Uses buttons, 
but typical 
textual list) 


Used, but less 
than artifact 
lists. 


Rollover activity 
rationale guides 


Triggered 


Optional 


None 


None 


Textual 


No indications 
that they were 
used. 


Problem 
development 
"Info” guides 


Triggered 


Optional 


None 


Problematic: 
Non-descriptive 
labels on trigger 
buttons 


Textual 


No indications 
that they were 
used. 


"Conductor” 

metaprocess 

map 


Triggered 


Optional 


None 


Problematic: 

• Confusing trigger 
button 

• Confusing layout 


• Graphical 
(Map) 

• Textual 


Not used very 
often. 
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